-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mapr deprecation #441
base: master
Are you sure you want to change the base?
Mapr deprecation #441
Conversation
Build failures:
|
I believe the primary failure is actually during the
#440 contains the proposal to address this issue. Re-expressing the next steps that need to happen: 1- ome/ansible-role-iptables-raw#13 needs to be reviewed, merged and released |
Seb: see https://github.com/ome/ansible-role-nginx-proxy/blob/1a2eb446951fe831126128c747beb6221d142047/templates/nginx-confd-proxy.j2#L50 for nginx setup. "Could put 410.html under `about" or "static" locations. |
As discussed this morning in answer to my question above about where to put:
It seems that the equivalent 50x handling is part of this ansible role: However, it's not clear how/where to add the custom This is somewhat similar to the 'maintenance page' we have specified at However, to show a similar page for the So, the options are:
So, probably best to go for option 1. This may not be precisely the correct response, but it will be most user-friendly and easiest for us, bearing in mind it would be nice to release this next week or soon after. |
Build is green now 👍 . |
@will-moore with #440 in, can you merge |
@sbesson Done |
Deployed to TASK [ome.nginx_proxy : nginx | proxy cache config] ****************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-cache.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmp3t9v16lg/nginx-confd-proxy-cache.j2
@@ -5,7 +5,6 @@
proxy_cache_path /var/cache/nginx/omerorender levels=1:2 keys_zone=omerorender:35m max_size=40g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omerometadata levels=1:2 keys_zone=omerometadata:65m max_size=25g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omeroapi levels=1:2 keys_zone=omeroapi:320m max_size=10g inactive=180d use_temp_path=off;
-proxy_cache_path /var/cache/nginx/omeromapr levels=1:2 keys_zone=omeromapr:100m max_size=5g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omeroviewers levels=1:2 keys_zone=omeroviewers:1m max_size=1g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/grafana levels=1:2 keys_zone=grafana:1m max_size=100m inactive=1m use_temp_path=off;
@@ -18,7 +17,6 @@
"~web(client|gateway)/get_thumbnail*" 0;
"~(webclient/)?api/*" 0;
"~static/*" 0;
- "~mapr/*" 0;
"~gallery-api/*" 0;
"~webclient/img_detail/*" 0;
"~iviewer/*" 0;
@@ -46,7 +44,6 @@
"~web(client|gateway)/render_*/*" omerorender;
"~webclient/metadata_*" omerometadata;
"~(webclient/)?api/*" omeroapi;
- # "~mapr/*" omeromapr;
"~webclient/img_detail/*" omeroviewers;
"~iviewer/*" omeroviewers;
"~grafana/*" grafana;
changed: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy cache config] ****************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | check websockets configuration] ****************************************************************************************************************
skipping: [test125-proxy] => (item={'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*', 'server': 'http://omeroreadonly', 'cache_validity': '1d', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omeromapr', 'location': '~ /mapr/*', 'server': 'http://omeroreadonly', 'cache_validity': '180d', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadonly', 'cache_validity': '1d'})
skipping: [test125-proxy] => (item={'name': 'omero', 'location': '/', 'server': 'http://omeroreadonly', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omerowebsockets', 'location': '= /omero-ws', 'server': 'http://omeroreadonlywebsockets', 'websockets': True, 'websocketsonly': True, 'read_timeout': 86400})
skipping: [test125-proxy] => (item={'name': 'grafanarender', 'location': '^~ /grafana/', 'server': 'http://192.168.2.92:3000/render/dashboard/db/', 'cache_validity': '1m'})
skipping: [test125-proxy] => (item={'name': 'prometheusfederate', 'location': '^~ /prometheus/federate', 'server': 'http://192.168.2.92:9090/federate', 'cache_validity': '15s'})
skipping: [test125-proxy] => (item={'name': 'searchengine', 'location': '^~ /searchengine', 'server': 'http://192.168.2.197:5577/searchengine', 'host_header': '$host/searchengine'})
TASK [ome.nginx_proxy : nginx | proxy websockets config] ***********************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy websockets config] ***********************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | check redirect configuration] ******************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy redirects config] ************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-redirect.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpjl5hj5mc/nginx-confd-proxy-redirect.j2
@@ -14,8 +14,4 @@
~/pgpc /webclient/?show=screen-1151;
/study/idr0083/figure/4i /webclient/img_detail/9822151/?dataset=10201&x=34105&y=84808&zm=25&c=1|318:9927$808080&m=g;
/study/idr0083/figure/4r /webclient/img_detail/9822152/?dataset=10201&x=80560&y=77440&zm=66&c=1|495:9204$808080&m=g;
- /mapr/api/antibody /searchengine/apidocs/;
- /mapr/api/compound /searchengine/apidocs/;
- /mapr/api/gene /searchengine/apidocs/;
- /mapr/api/config /mapr/api/config;
}
changed: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy redirects config] ************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream-conf.d directory] ***********************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream proxy config] ***************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream proxy config] ***************************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy config] **********************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpnatinmsu/nginx-confd-proxy.j2
@@ -12,7 +12,9 @@
ssl_certificate /etc/ssl/localcerts/bundled.crt;
ssl_certificate_key /etc/ssl/localcerts/server.key;
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # don't use SSLv3 ref: POODLE
+ ssl_protocols TLSv1.2 TLSv1.3; # don't use SSLv3 ref: POODLE
+ ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305;
+
#charset koi8-r;
#access_log /var/log/nginx/log/host.access.log main;
@@ -44,9 +46,8 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
-
location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 /searchengine/apidocs/;
+ return 302 $redirect_uri;
}
location = /webclient {
changed: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
--- before: /etc/nginx/conf.d/proxy-cachebuster.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmp2pvaxxro/nginx-confd-proxy.j2
@@ -33,6 +33,9 @@
port_in_redirect off;
location ~ ^/(mito|tara|pgpc|study)($|/) {
+ return 302 $redirect_uri;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
return 302 $redirect_uri;
}
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
--- before: /etc/nginx/conf.d/proxy-idr-demo.openmicroscopy.org.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpifpytgda/nginx-confd-proxy.j2
@@ -10,7 +10,9 @@
ssl_certificate /etc/ssl/localcerts/bundled.crt;
ssl_certificate_key /etc/ssl/localcerts/server.key;
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # don't use SSLv3 ref: POODLE
+ ssl_protocols TLSv1.2 TLSv1.3; # don't use SSLv3 ref: POODLE
+ ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305;
+
#charset koi8-r;
#access_log /var/log/nginx/log/host.access.log main;
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []}) |
@sbesson - mostly that diff looks as expected (but not at
I had manually added this myself by editing of /etc/nginx/conf.d/proxy-default.conf:
so it looks like the diff is comparing my manual changes here to the changes after your update? But I also don't understand in
This looks like removal of redirects to |
That's correct, this is capturing the changes was applied to
Clearly someone edited this file as these changes are neither on If you don't want to use the redirect map, then you probably need to put your 302 redirect under |
Ah - understood! Done, thanks. |
After running the TASK [ome.nginx_proxy : nginx | proxy config] **************************************************************************************************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-67117nlkwjgz4/tmp4odhrluu/nginx-confd-proxy.j2
@@ -46,9 +46,6 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
- location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 $redirect_uri;
- }
location = /webclient {
return 302 /webclient/?experimenter=-1;
@@ -76,6 +73,9 @@
}
location = /submitter-survey {
return 302 https://forms.gle/beR4vqLrgLgRzgSX9;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
+ return 302 /searchengine/apidocs/;
}
location ~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/* {
changed: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
--- before: /etc/nginx/conf.d/proxy-cachebuster.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-67117nlkwjgz4/tmp4y0krw4k/nginx-confd-proxy.j2
@@ -35,9 +35,6 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
- location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 $redirect_uri;
- }
location = /webclient {
return 302 /webclient/?experimenter=-1;
@@ -65,6 +62,9 @@
}
location = /submitter-survey {
return 302 https://forms.gle/beR4vqLrgLgRzgSX9;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
+ return 302 /searchengine/apidocs/;
}
location ~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/* {
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
ok: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []}) |
@sbesson Thanks - that looks like it's working fine now 👍 . |
👍 Do I understand correctly that this PR is now only waiting on a release of the deployment/ansible/group_vars/omero-hosts.yml Line 245 in cb12a87
|
Yes, I think so |
I'll wait for idr-gallery release (and mapr too) then bump them both in this PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The components and NGINX changes are consistent with the discussions around the deprecation of the mapr endpoints from the last few weeks and have been successfully deployed at test125
.
TASK [ome.omero_web : omero web | install requirements] *************************************************************************************************************************************************
changed: [test125-omeroreadonly-2]
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-3]
...
TASK [ome.omero_web : omero web | flush systemd handlers] ***********************************************************************************************************************************************
RUNNING HANDLER [ome.omero_web : omero-web rewrite omero-web configuration] *****************************************************************************************************************************
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-3]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-2]
RUNNING HANDLER [ome.omero_web : omero-web restart omero-web] *******************************************************************************************************************************************
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-2]
changed: [test125-omeroreadonly-3]
Approving from a deployment perspective and assigning a few reviewers for the functional testing before we apply this on prod125
Testing on idr-testing looks good for me. Search, redirects etc. |
Tested on idr-testing. Looks good. Search, redirects and pop-up working as expected. |
Clicking on |
Still need to find why Looking for "sapiens" etc in ngninx conf finds nothing. Looking for mapr finds some things...
The cache itself is stored at Try
Now, this redirects correctly... Try to find weblient page with "Homo sapiens" in caches. Use Found 1 under
Other dirs either don't find anything or take too long to grep:
|
Ah-ha! After waiting a couple of hours, https://idr-testing.openmicroscopy.org/mapr/organism/?value=Homo%20sapiens has stopped returning a cached page and now re-directs to idr-gallery search as intended! NB: Let's test some more on idr-testing with the EDIT: Done a fair bit of testing and everything seems to be working, apart from a couple of "no results" - issue created at ome/omero_search_engine#104 |
@pwalczysko Can you confirm that clicking on |
Confirmed. FF on Mac M1, new private window. Click on Homo sapiens goes to search https://idr-testing.openmicroscopy.org/search/?key=Organism&value=Homo+sapiens&operator=equals |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did some more extensive testing on idr-testing just now. LGTM
From today's conversation in OME Slack
Were the NGINX changes applied to deployment/ansible/group_vars/proxy-hosts.yml Lines 67 to 72 in cb12a87
|
No.
I don't know about any playbook, but sounds like a yes. Anyway the mapr caching should already be disabled by this change in this PR:
|
These changes have been deployed since last week (#441 (review)). My understanding is that we had clearly established that these changes alone were not sufficient to since the Following the Monday discussion, two changes were performed on
113,119c118,124
< proxy_cache $cache_zone_name;
< proxy_cache_key $cache_key;
< proxy_cache_valid 200 180d;
< proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
< proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
< proxy_cache_background_update on;
< proxy_cache_bypass $cache_refresh;
---
> #proxy_cache $cache_zone_name;
> #proxy_cache_key $cache_key;
> #proxy_cache_valid 200 180d;
> #proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
> #proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
> #proxy_cache_background_update on;
> #proxy_cache_bypass $cache_refresh; I just reverted the latter configuration changes and restarted NGINX on |
There was also a single cache that showed up in my grepping for home page html that I renamed (see above):
This could have been cached in some custom step somewhere and might be responsible for the custom behaviour of I can confirm that EDIT: ...and also I'm not seeing anything matching that page under
|
@sbesson when you say "two changes were performed on test125", I understand the first (which I did) but not:
Do you know what caused that? I don't think I did that (unless by some mechanism I don't understand)?! |
Apologies for the confusion, clearly the communication did not go through. I applied this change manually on
Absolutely, this change was applied manually on top of the configuration set by the state of this PR so we need another change in addition to what is suggested here. As mentioned in #441 (comment), my suspicion is that we want to remove deployment/ansible/group_vars/proxy-hosts.yml Lines 67 to 72 in cd682d1
|
Yes please! Go ahead, thanks. |
With diff --git a/ansible/group_vars/proxy-hosts.yml b/ansible/group_vars/proxy-hosts.yml
index 8a6cd5d..d46d9fc 100644
--- a/ansible/group_vars/proxy-hosts.yml
+++ b/ansible/group_vars/proxy-hosts.yml
@@ -64,12 +64,6 @@ _nginx_proxy_backends_omero:
cache_validity: 1d
maintenance_flag: "{{ omero_maintenance_flag }}"
maintenance_uri: "{{ omero_maintenance_uri }}"
-- name: omeromapr
- location: ~ /mapr/*
- server: http://omeroreadonly
- cache_validity: 180d
- maintenance_flag: "{{ omero_maintenance_flag }}"
- maintenance_uri: "{{ omero_maintenance_uri }}"
- name: omerostatic
location: ~ /static/*
server: http://omeroreadonly I have the following changes in dry-run mode TASK [ome.nginx_proxy : nginx | proxy config] ***********************************************************************************************************************************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-6847698rc123m/tmpzfgu325k/nginx-confd-proxy.j2
@@ -86,38 +86,6 @@
proxy_cache $cache_zone_name;
proxy_cache_key $cache_key;
proxy_cache_valid 200 1d;
- proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
- proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
- proxy_cache_background_update on;
- proxy_cache_bypass $cache_refresh;
- proxy_no_cache $skip_cache;
-
-
- proxy_cache_lock on;
- proxy_cache_lock_age 1m;
- proxy_cache_lock_timeout 1m;
-
- proxy_ignore_headers "Set-Cookie" "Vary" "Expires";
- proxy_hide_header Set-Cookie;
-
-
-
- if (-f /srv/www/omero-maintenance.flag) {
- return 503;
- }
- error_page 503 /about/omero-maintenance.html;
-
-
- }
-
- location ~ /mapr/* {
-
- proxy_pass http://omeroreadonly;
- proxy_redirect http://omeroreadonly $scheme://$server_name;
-
- proxy_cache $cache_zone_name;
- proxy_cache_key $cache_key;
- proxy_cache_valid 200 180d;
proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on; Should I try and execute this outside check-mode? |
@sbesson - yes, great. Let's give it a test, thanks |
The change above is now deployed on TASK [ome.nginx_proxy : nginx | proxy config] ***********************************************************************************************************************************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-70170hrpvvmze/tmp4v98_6zl/nginx-confd-proxy.j2
@@ -86,38 +86,6 @@
proxy_cache $cache_zone_name;
proxy_cache_key $cache_key;
proxy_cache_valid 200 1d;
- proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
- proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
- proxy_cache_background_update on;
- proxy_cache_bypass $cache_refresh;
- proxy_no_cache $skip_cache;
-
-
- proxy_cache_lock on;
- proxy_cache_lock_age 1m;
- proxy_cache_lock_timeout 1m;
-
- proxy_ignore_headers "Set-Cookie" "Vary" "Expires";
- proxy_hide_header Set-Cookie;
-
-
-
- if (-f /srv/www/omero-maintenance.flag) {
- return 503;
- }
- error_page 503 /about/omero-maintenance.html;
-
-
- }
-
- location ~ /mapr/* {
-
- proxy_pass http://omeroreadonly;
- proxy_redirect http://omeroreadonly $scheme://$server_name;
-
- proxy_cache $cache_zone_name;
- proxy_cache_key $cache_key;
- proxy_cache_valid 200 180d;
proxy_cache_methods GET HEAD; # Only GET and HEAD methods apply
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
changed: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
ok: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
ok: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []})
TASK [ome.nginx_proxy : nginx | proxy upstream servers] *************************************************************************************************************************************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | create proxy ssl certificate directory] *********************************************************************************************************************************************************************************************************************
ok: [test125-proxy] => (item={'nginx_proxy_ssl_certificate': '/etc/ssl/localcerts/bundled.crt'})
skipping: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
skipping: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
skipping: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []})
TASK [ome.nginx_proxy : nginx | create proxy ssl certificate key directory] *****************************************************************************************************************************************************************************************************************
ok: [test125-proxy] => (item={'nginx_proxy_ssl_certificate_key': '/etc/ssl/localcerts/server.key'})
skipping: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
skipping: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
skipping: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []})
TASK [ome.nginx_proxy : nginx | copy proxy ssl certificate] *********************************************************************************************************************************************************************************************************************************
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | copy proxy ssl certificate key] *****************************************************************************************************************************************************************************************************************************
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy] => (item=None)
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | start service] **********************************************************************************************************************************************************************************************************************************************
ok: [test125-proxy]
RUNNING HANDLER [ome.nginx : restart nginx] *************************************************************************************************************************************************************************************************************************************************
changed: [test125-proxy]
PLAY RECAP **********************************************************************************************************************************************************************************************************************************************************************************
test125-management : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-omeroreadonly-1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-omeroreadonly-2 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-omeroreadonly-3 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-omeroreadonly-4 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-omeroreadwrite : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test125-proxy : ok=44 changed=2 unreachable=0 failed=0 skipped=17 rescued=0 ignored=0
test125-searchengine : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 |
Bit of testing this morning - all seems to be working fine 👍 - no caching issues etc. |
Additional changes required - see #441 (comment)
That's the change above at #441 (comment) included in the last commit. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This matches the state of the test125
deployment. Once the functional behavior has been validated, this should be ready to deploy onto prod125
.
@jburel could you have a look at this (it's deployed on idr-testing) as we'd like to get this merged at the end of this week or start of next week (I'm off tomorrow and Friday). Thanks! |
- idr-gallery==3.11.0 | ||
- omero-mapr==0.5.3 | ||
- omero-iviewer==0.15.0 | ||
- idr-gallery==3.12.1 | ||
- omero-figure==6.2.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Figure is not used but some libraries bumped happened since and it could potential lead to some dependencies issues down the line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can see one of three options:
- do nothing and capture as an issue
- bump OMERO.figure - if so to which version?
- remove OMERO.figure
Which one would you recommend?
This represents the changes I have been making manually on idr-testing with the replacement of mapr searches with searchengine, described at IDR/idr-gallery#45
Summary of changes:
/mapr/gene/
get handled by idr-gallery instead of mapr/mapr/api/[key]/
requests we now response with a 410 response. [EDIT] Now redirects to /searchengine/apidocs/ 302 response - see below.NB: I couldn't see where to add the file I created with:
sudo vi /usr/share/nginx/html/410.html
(based on the 50x.html)Also the config for this file looks like:
In my testing, I had added this in
proxy-default.conf
right before thelocation
entry that I have added above:The
error_page 410
about follows on fromerror_page 50x
etc section in the same file, but I don't see that this is specified anywhere in this repo, so maybe that comes from existing nginx config somewhere?